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Chapter 2: Fault isolation 


This chapter helps you to identify problems that might occur while using the 
Meridian IVR Release 2.0/I system and related devices. 


The procedures described in this chapter may produce results requiring a 
specific maintenance procedure. In these cases, you are directed to a specific 
procedure in Chapter 4, “Maintenance activities”. 


If a fault persists, contact your Nortel service representative. 


Gathering information 


To diagnose a system failure effectively, you should determine which 
processes were active at the time of the failure. 


Once the system has been reset, the state of the system at the time of the 
failure is lost. 


You can check the status of all Meridian IVR processes by issuing the 
following UNIX command: 


ps -ef | grep ‘\.\/’ 


With this command, ps displays the process table, and grep limits the display 
to those Meridian IVR processes with UNIX path names with the special 
character ./. 


Meridian IVR requires certain processes to be running at all times. Other 
Meridian IVR processes may run but are not considered essential. 
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The following table lists the processes for Meridian IVR: 


Table 2-1 


Meridian IVR processes 





Process name 


Process description 





gui_main 


Main fax gui 





gui_sys 


Fax system administration 





gui_mdm 


Fax modem administration 





gui_mon 


Fax modem monitor 





gui_cal 


Call back administration 





gui_log 


OM and log administration 





gui_ve 


View and edit administration 





frusr 


Fax user function 





fax_server 


Fax server process 





log_server 


Log server process 





cal_server 


Call back server process 





ses_server 


One point for each channel 





gs 


PostScript conversion 





img_create 


Image create process 





cal_submit 


Call back submit process 





img_print 


Image print process 





dcat 


Allows the dynamic mixture of variable and fixed data 
from files and text buffers as part of a Meridian IVR 
call flow. 





scm 


System configuration manager 





sai 


System application interface 





xai 


System monitor 





sde 








System database editor 
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Table 2-1 
Meridian IVR processes (Continued) 





Process name 


Process description 





























xae Graphical application editor 
xconsole Main error console 

xclock Clock 

mwm X manager for IVR 

passwd Password change process 
xbup Backup and restore utility 
ntcu MMconsole process 
accdiag Access diagnostic process 
uif IVR main gui process 





mcm (parent) 


Meridian channel manager 





mem (child) 


Meridian channel manager 





vrtd (parent 1) 


Voicetek run-time daemon 





vrtd (parent 2) 


Voicetek run-time daemon 





vrtd (child x 4) 


Voicetek run-time daemon 
































psm Protocol state machine 
ueh User error handler 

vtk VTK request router 

vip VTK interface process 
vrm VTK resource manager 
cli Call logic interpreter 

dbs Voicetek database server 
qds Query DBMS server 

csc Call scheduler 
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Table 2-1 


Meridian IVR processes (Continued) 





Process name 


Process description 





sad 


System administration daemon 





vit 


File transfer 





pmg 


Pull manager 





trs 


Host connectivity session manager 





trsl 


Trs log process 





msg (multi) 


Messaging process 





trsc 


TRS protocol handling process 





cli_s 


Call logic process 





ust 


User statistic process 





Ih 


Access link handler 





Ihrx 


Link handler 





nedit 


Report writer 





sri_disp 


Report writer related process 





frep 


Report writer related process 





vpe 


Voice prompt editor 





sam 


System application monitor 





csnap 


Curses based snap 





rstat 


Reset statistic 





sched 


Schedule application process 





devent 


Display sched application 





qds 


Query DBMS server 





xdx 


Database process 








Xpowerchute 





UPS gui interface 
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Table 2-1 
Meridian IVR processes (Continued) 
Process name Process description 
_xpwrchute UPS gui interface 
_upsd UPS daemon 
_upsd UPS daemon 
agent Apertus agent process 
cdmserver Apertus process 
executive Apertus software daemon 
watchdog Apertus software daemon 
express Express manager process 
CM Configuration utility process 
CFG Control communications and diagnostics process 
End 











Examining interprocess communication queues 


The interprocess communication (IPC) queues are used by Meridian IVR to 
let its processes communicate with each other. If these processes do not 
function properly, the system will fail. To view the queues in the system, 
issue the following command: 


ipcs -a 


This command displays the last sender, the last receiver, and the number of 
bytes in each queue. 


During normal operation, the queues contain zero bytes because they are 
being read from and emptied constantly. 


If the queues are not empty, one process is no longer reading information 
from the queues. Since there are limited system resources for each individual 
queue and for all combined queues, the system will fail. You must determine 
which process has stopped. 
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Meridian IVR log files 


The following error logs are found in Meridian IVR: 


/a/ivr/log.d/event.log 
Transaction log file 


/a/ivr/log.d/start.log 
Startup log file 


/a/ivr/log.d/xconsole.log 
Xconsole log file 


/u/fax/log/yyyymmdd.log 
Fax event log 


/a/fax/log/cal_server.err 
Fax error log 


/u/fax/log/fax_server.err 
Fax error log 


/u/fax/log/log_server.err 
Fax error log 


/a/admin/accdiag.log 
ACCESS diagnostics log 


/a/ivr/vrs/exe/accdiag.log 
ACCESS diagnostics log 


/a/ivr/vrs/exe/lhn.log 
Link handler log file 


/a/ivr/vrs/exe/lhn_old.log 
Old (previous) link handler log file 


/a/ivr/vrs/log.d/start.log 
VRS start log (logs last start time of VRS) 


/a/ivr/vrs/log.d/event.log 
VRS event log 
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Examining the error logs 


The error logs contained in /u/ivr/log.d/event.log (viewing the Reports 
Generator, this file is referred to as the Transaction log file) and 
/u/ivr/vrs/log.d/event.log are important sources of system information. These 
logs report error messages for each process which are used for debugging 
applications and system failures. 


Use the more command to examine the contents of the error logs. 


Logs are circular files of 500 lines in length. If any logs contain pertinent 
information, save the file to a unique name. 


Note: A circular file has a set length. When messages exceed this length, 
they are written at the beginning of the file and, thus, overwrite the 
earliest messages in the file. 


Examining the /tmp directory 


The /tmp directory contains information about each process in the system. 
Each process has a file called pid.msg.log in the /tmp directory. 


Process shutdown information 
Each file indicates the last state of the process which created it. These files 
can often indicate which process shutdown abnormally. 


Examining the Meridian IVR console window 


The Meridian IVR console window displays Meridian IVR status errors at 
start-up and shutdown. 


These messages are also written to a file. 


Security problems are also displayed in this console window. 


Reading Transaction Log Report self-diagnostic messages 


Whenever Meridian IVR is powered up, it performs self-diagnostics and then 
reports the results of these diagnostics to the system console. 


One item is Message #102 which reports a CCC1 or a CCCO diagnostic reply 
in the Transaction Log Report. 
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CCCo - Fatal errors 


If Meridian IVR has fatal errors, diagnostics report them in a CCCO reply. 
(Fatal errors are errors that prevent Meridian IVR from running.) 


CCC1 - Non-fatal errors 


If Meridian IVR has non-fatal errors, diagnostics report them with one or 
more CCC1 replies. 


To read CCC1 and CCCO replies, go to the Transaction Log Report and look 
at every Message #102 that has been added to the report since the last 
diagnostics were performed. 


Meridian Access link setup problem 


If you suspect a Meridian Access link setup problem, you can check Meridian 
Access link settings by following the procedure below. 


Procedure 2-1 
Checking Meridian Access link settings 


1 Check that Meridian Access link is enabled on Meridian Mail by going 
to the Meridian Mail menu and selecting General System 
Administration. 


2 Select General System Options. If the Meridian link is enabled, it is 
listed in the Available Features display. 


Check that the Access link cable is connected to data port MMLINK. 


4 View settings of the hardware database by selecting Hardware 
Administration on the Meridian Mail menu. 


5 Select Data Port Configuration to display a list of configured system 
data ports. One data port should appear as MMLINK. 


MMLINK must be set to the proper baud rate (for example, 9600 for 
Meridian Mail 8) and a specific data port location. 


For more information on how to establish the Meridian Access link with IVR, 
see the Meridian IVR Installation Guide (NTP 555-9001-210). 
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Troubleshooting operational problems 


This section provides general guidelines for troubleshooting operational 
problems in the Meridian IVR system. 


If you cannot correct the problem, contact your Nortel service representative. 


Failure to synchronize the Meridian Access link 


The Meridian Access link manages communications between Meridian IVR 
and Meridian Mail. 


Use the MM Link Diagnostics menu on your terminal to check that 
communications are synchronized and running properly. This menu displays 
link handler status. 


If the Link not synchronized message appears, do the following: 


1 


Check if the Meridian Access link is installed using the tools level in 
Meridian Mail and the MMverify command (option 13, for example, 
Link Diagnostics). 


Check that the EC/RSM utility card is seated properly within the 
Meridian Mail system. 


Check that the fanout cable at the RSM utility card is connected 
properly. 
Check the I/O panel cabling. 


Check that the Digiboard cable is connected to the first port of the 
RSM/EC fanout cable. 


If the fanout cable is connected to the wrong RSM/EC utility port, you 
might have to reset Meridian Mail to synchronize the Meridian Mail link. 


An application cannot run 


If the Meridian IVR core software is running but an application is not 
executing, check system configuration parameters using the system 
configuration interface. 
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Procedure 2-2 
Checking system configuration parameters 


1 


Check the mailbox id and the channel mailbox password. Correct if 
necessary. 


Start Meridian IVR, and, through the system administration menu, 
display the system monitor administration window. 


Press the Node Toggle Button to display channel activity. 


e If Meridian Mail channels are all gray, check that the Meridian Mail 
link is connected and synchronized. 


e If the number of configured channels exceeds the number of 
defined channels, only the defined channels will work. 


* The number of channels on the screen should match the number of 
channels purchased or defined for the system. 


Check the log files and examine any error messages. 


On the system administration menu, select system configuration to 
display the system configuration screen. Check the number of 
configured channels. 


The system configuration screen shows the Number of Nodes and 
Maximum # of System Wide Prompts. 


The number of nodes should always be 1. 


Meridian IVR cannot run 


If the Meridian IVR core software is not running check the following error 


logs: 


/a/ivr/log.d/event.log 


/a/ivr/log.d/xconsole.log 


If you cannot correct the problem, contact your Nortel service representative. 


A call cannot reach Meridian IVR 


If a call cannot reach a Meridian IVR channel, check the system configuration 
on Meridian IVR and on Meridian Mail. 


If the configuration has changed, reset Meridian IVR as described in the 
Meridian IVR Installation Guide (NTP 555-9001-210). 
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Failure to access the voice prompt mailbox 


If Meridian IVR is running and all of its channels are defined as dedicated, 
then the VPE will not have a free channel for logging in to the voice prompt 
mailbox. The VPE needs at least one unused shared channel on the Meridian 
Mail System to access the mailbox. 


To check if the VPE has a shared channel, follow these steps: 





CAUTION 
Do not stop Meridian IVR during the recording of 
l voice prompts because this causes the Meridian Mail 
link to stop. When the link is stopped, changes made 
during the VPE session cannot be saved. 











Check that the mailbox which the VPE is trying to access is available and that 
the password being used is correct. 


Meridian Mail console window not available 


An optional Meridian Mail console window can be displayed on the SVGA 
monitor using special cabling. If no Meridian Mail console window appears 
on the SVGA monitor, check that the cabling between the Meridian Mail I/O 
panel and the Meridian IVR panel is the correct cable. Refer to the Meridian 
IVR Installation Guide (NTP 555-9001-210). 


The Meridian Mail console baud rate must be 2400 bps to work properly with 
IVR. 


Contact the Nortel service representative to correct this problem. 
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